Outline

1. Part 1: Understanding the Multiverse
   

2. Part 2: MB2-P Pupillometry Case Study

3. Co-authors:
   Melanie S. Schreiner • Alvin Tan • Marlena Mayer • Robert Hepach and the MB2-P Team —

What is Multiverse Analysis?

• “A systematic exploration of all reasonable analytical paths” (Steegen et al., 2016)
  
• Enhances transparency and reduces researcher degrees of freedom
  
• A garden of analytical forking paths: grounded in theory

Why It Matters for Developmental Science

• Universal Applicability: works across contexts and methods
  
• Flexible Implementation: supports exploratory and confirmatory designs
  
• Credibility Enhancement: addresses replication challenges
  
• ManyBabies Alignment: fits with goals of transparency and rigor

The Multiverse Landscape

• Data Inclusion/Exclusion: who/what to include?
  
• Preprocessing Methods: cleaning, transforming
  
• Variable Operationalization: defining constructs
  
• Statistical Models: which methods and why

Visualizing the Multiverse

Specification Curve

Volcano Plot

Volcano Plot

Study Design Overview

• Cross-lab pupillometry with over 30 labs worldwide
• Age cohorts: toddlers and adults
• Condition × Outcome × Time modeling
• From: Calignano, Girardi, Altoè., 2024 First steps into the pupillometry multiverse of developmental science. Behav Res Methods.

ManyBabies 2: Theory of Mind in Infancy

MB2 – Do toddlers and adults engage in spontaneous Theory of Mind (ToM)?

MB2 – ToM: Paradigm

MB2 – ToM

MB2 – ToM

MB2P – Secondary Data Analysis

MB2P – Secondary Data Analysis

MB2P – Secondary Data Analysis

• MB2P investigates responses to goal-congruent vs. goal-incongruent outcomes
• Is the child surprised if the protagonist (bear) acts incongruently with the goal (following the mouse)?
• Measures: pupil dilation and looking time during events

The MB2P Preregistration and the Multiverse Approach

The MB2P Preregistration and Data Simulation

• Data Collection by MB2 completed in 2023. Raw data will be shared after Stage 1.
• Analyses will focus on the second test trial.
• Data Simulation mirrors expected data structure to pre-register processing and analysis strategies.
• Simulated dataset includes:
  • Participant ID, Two age cohorts , Timestamps or durations, x/y coordinates, Pupil size (left/right), Lab ID, Conditions and outcomes

MB2-P Multiverse Structure

• Five Forking Dimensions:
  • Pupil plausibility
  • Gaze within screen
  • Filtering via moving average
  • Baseline correction window (1s, 0.5s, 0.25s)
  • Participant exclusion (none, trial 2, trial 1&2)

Simulated Data

Density Plot by Condition × Outcome × Age

DF1: Pupil Plausibility

DF1: Pupil Plausibility

DF2: Gaze Filtering

DF2: Gaze Filtering

DF3: Moving Average Filter

DF3: Moving Average Filtering

DF4: Baseline Correction

`geom_smooth()` using formula = 'y ~ x'

DF4: Baseline correction

DF5: Participant Inclusion Criteria

# A tibble: 2 × 2
  df5          n
  <chr>    <int>
1 excluded 47200
2 included   800

Specification Curve of the Multiverse’ Models

Specification Curve of Model Estimates

Volcano plot of the Multiverse’ Models

Volcano Plot of Effect Estimates

Thank You

• Questions? Feedback?